1. Statement of the Technical Field
The inventive arrangements relate to teleoperation systems, and more particularly to teleoperation systems that operate in a high latency environment.
2. Description of the Related Art
Teleoperation involves the manipulation and control of objects and devices over long distances. A teleoperation system usually includes an unmanned robotic system and a user control console. A data link facilitates communications between the control console and the unmanned robotic system so that user commands can be provided to the unmanned robotic system which is usually located some distance away. Most unmanned robotic systems are controlled over a wireless link. Unmanned remotely controlled systems include unmanned aerial vehicles (UAV), unmanned ground vehicles (UGV), unmanned underwater vehicles (UUV) and robotic manipulators. An unmanned remotely controlled system as described herein can sometimes include a movable vehicle portion (e.g. a tracked base unit in the case of a UGV) and a robotic arm with a grasping element for manipulating objects in the field.
A control console for an unmanned robotic system will generally include some type of user input device for receiving user control inputs. The user input device can take many forms. A typical arrangement can include a joystick or movable grip handle which the user can urge in various directions to cause appropriate movements or actions of the unmanned robotic system. In systems that include a movable vehicle portion and a robotic arm portion, the user input device can be used to control the vehicle position, robot arm position, or both. If the robotic arm includes a grasping device, the user input device can include a suitable control element for opening and closing mechanical fingers provided on the grasping device.
An imaging system is typically provided to facilitate remote control operation of an unmanned robotic system. The imaging system will usually include a camera mounted to a portion of the unmanned robotic system and a display unit located at the operator console. The system is arranged so that live motion images (video) collected by the camera is communicated using the data link to the operator console, where the live motion images are then presented to the user on the display unit. In an ideal scenario, the live motion images will be communicated to the user display on a near instantaneous basis so that the user will always have an accurate visual basis for understanding the position or pose of the robot relative to its environment. However, due to data link latency and low bit rates this ideal scenario is not always realized. In fact, in scenarios where the data link is operating at very low bit rates camera feedback to the operator can be relatively infrequent. For example, in some scenarios, a new image may be presented to the operator only once every five seconds.
When data link latency is substantial it can lead to a mismatch as between user intent and resulting robot motion. Consider that input commands from a user are generally based on feedback from the robot environment in the form of the live motion images and telemetry as described herein. In situations where high data link latency is present the visual imagery and telemetry data presented to a user at the control console may not represent the actual current operating state of the unmanned robotic system. More particularly, the information presented to the user in such a scenario can be latent imagery and latent telemetry communicated from the robotic system at an earlier time. The latency of this feedback data means that it may no longer accurately represent a current pose or state of the robotic system. Such latent imagery can therefore cause a user to misunderstand the control inputs that are required to achieve a desired end. Ultimately, latent feedback data as described herein will negatively impact teleoperation, potentially causing damage to the robotic system and/or objects in the robot environment.